Skip to content

Validate arrow_cast fixed-size binary length#22297

Open
Sean-Kenneth-Doherty wants to merge 1 commit into
apache:mainfrom
Sean-Kenneth-Doherty:codex/arrow-cast-fixed-size-binary
Open

Validate arrow_cast fixed-size binary length#22297
Sean-Kenneth-Doherty wants to merge 1 commit into
apache:mainfrom
Sean-Kenneth-Doherty:codex/arrow-cast-fixed-size-binary

Conversation

@Sean-Kenneth-Doherty
Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

arrow_cast(NULL, 'FixedSizeBinary(-1)') currently parses the type string successfully and can later panic in Arrow buffer allocation with a LayoutError. DataFusion should reject that invalid Arrow type specification before planning/simplification rewrites the function into a cast.

What changes are included in this PR?

  • Adds shared arrow_cast type parsing/validation after Arrow DataType parsing.
  • Rejects negative FixedSizeBinary element lengths with a DataFusion execution error.
  • Reuses the same validation for arrow_try_cast so return-field planning and simplify-time rewrites agree.
  • Checks nested types too, so List(FixedSizeBinary(-1)) is rejected before it reaches execution.
  • Adds SQL logic regressions for arrow_cast, arrow_try_cast, and nested FixedSizeBinary(-1).

Are these changes tested?

Yes.

  • cargo test -p datafusion-functions arrow_cast (compiled successfully; no tests matched the filter)
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- arrow_typeof.slt
  • cargo test -p datafusion-functions
  • cargo fmt --check
  • cargo clippy -p datafusion-functions --all-targets -- -D warnings
  • git diff --check

Are there any user-facing changes?

Yes. Invalid FixedSizeBinary(-1) type specs passed to arrow_cast / arrow_try_cast now return a normal DataFusion execution error instead of allowing a later panic.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels May 17, 2026
@Sean-Kenneth-Doherty
Copy link
Copy Markdown
Author

Local validation on 5ba2d053:\n\n- cargo test -p datafusion-sqllogictest --test sqllogictests -- arrow_typeof.slt -> completed 1/1 files\n- cargo test -p datafusion-functions arrow_cast -> compiled successfully; 0 matching unit tests, 282 filtered out\n- cargo test -p datafusion-functions -> 282 passed; doctests 3 passed, 4 ignored\n- cargo fmt --all --check -> passed\n- cargo clippy -p datafusion-functions --all-targets -- -D warnings -> passed\n- git diff --check origin/main...HEAD && git diff --check -> clean\n\nThe GitHub Process check is also green; no full matrix has reported yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

panic: arrow_cast(NULL, 'FixedSizeBinary(-1)') panics with LayoutError

1 participant